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Abstract 

In 2014, Massive Open Online Courses (MOOCs) are expected to witness a 
phenomenal growth in student registration compared to the previous years (Lee, 
Stewart, & Claugar-Pop, 2014). As MOOCs continue to grow in number, there has been 
an increasing focus on assessment and evaluation. Because of the huge enrollments 
in a MOOC, it is impossible for the instructor to grade homework and evaluate each 
student. The enormous data generated by learners in a MOOC can be used for 
developing and refining automated assessment techniques. As a result, "Smart 
Systems" are being designed to track and predict learner behavior while completing 
MOOC assessments. These automated assessments for MOOCs can automatically 
score and provide feedback to students multiple choice questions, mathematical 
problems and essays. Automated assessments help teachers with grading and also 
support students in the learning processes. These assessments are prompt, 
consistent, and support objectivity in assessment and evaluation (Ala-Mutka, 
2005). This paper reviews the emerging trends in MOOC assessments and their 
application in supporting student learning and achievement. The paper concludes by 
describing how assessment techniques in MOOCs can help to maximize learning 
outcomes. 

Keywords: MOOCs; automated assessments; smart systems; automated grading; 
automated feedback; learning and achievement; assessment and evaluation 
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I. Introduction 

In 2014, Massive Open Online Courses are expected to see a 100% rise in student registration (Lee, 
Stewart, & Claugar-Pop, 2014). This prediction comes two years after the New York Times labeled 
2012 as the "Year of the MOOCs". MOOCs continue to grow with more universities and institutions 
getting into the fray to offer these courses to learners from all around the world. While the three 
major MOOC providers - Coursera ( https://www.coursera.org ). Udacity ( https://www.udacitv.com ). 
and edX ( https://www.edx.org ) are based in the US, 2013 has been witness to the launch of startups 
in other countries - France based France Universite Numerique f http://www.france-universite- 
numerigue.fr) : FutureLearn (https://www.futurelearn.com) in the UK; Open2Study 

( https://www.open2studv.com ) in Australia, and iversity ( https://iversitv.org ) in Germany (Shah, 
2013). 

This paper contributes to the growing literature on the current assessment techniques and trends in 
MOOCs. The paper begins by providing a brief overview of the government support for MOOCs that 
has relevance to assessment and accreditation of MOOCs including the universalization of education. 
The paper then discusses MOOC taxonomy since it has either direct or indirect relevance to the 
assessment design and development in a MOOC. The paper provides case examples of MOOCs which 
have adopted automated assessment techniques including emerging tools and technology. Finally, the 
paper concludes how various MOOC assessments can help support student learning and achievement. 

II. Emerging Trends and MOOC Taxonomy 

The number of course offerings by the MOOC providers and student enrollments continue to grow 
even as the completion rates have been abysmal. Research has shown that the completion rates are 
below 10% (Jordan, 2013). The growing popularity of MOOCs makes it impossible to ignore the 
emerging trends in online education. The existing infrastructure and resources need to stay up-to- 
speed with the demands of online education (Lee, Stewart, & Claugar-Pop, 2014). As a result, 
universities and institutions are walking a tight rope and considering either extending, integrating or a 
total revamp, if necessary to accommodate this changing landscape in online education. The following 
section discusses two such latest government initiatives followed by a typology of MOOC instructional 
models. 

a. The US Department Initiative 

Besides universities and institutions, governments realize the need for supporting MOOCs with an aim 
to provide education for all. The US State Department launched the "MOOC Camp" to mentor MOOC 
participants (Haynie, 2014). The MOOC Camps are organized by US missions and consulates abroad. 
The camps include Fulbright scholars and embassy staff who lead discussions on content and topics for 
MOOC participants in countries abroad. The pilot version was run in 2013 and more courses will be 
offered to students in India, Peru, Tanzania, Nigeria with more locations to follow (Haynie, 2014). The 
MOOC Camps are a big step towards supporting MOOCs as a comprehensive effort not just to 
popularize MOOCs, but to also make them available to a wide range of learners around the world. The 
global outreach of MOOC will not only help in the universalization of education but sustained efforts 
from the government will also go a long way in evolving assessment and credentialing of MOOCs. 

b. ACE Credit Recommendation 

Recently, the American Council on Education (ACE) also recommended offering credit for five courses 
on the popular MOOC platform Coursera (Coursera, 2013). This is an important step in the formal 
acceptance and approval of MOOCs for learning and assessment. A bill in the California legislature also 
aims to offer credit for online courses that have been approved by the faculty (Gardner & Young, 
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2013). These decisions have been prompted as MOOC participants have grown phenomenally over the 
recent years. Top universities and institutions especially in the US and around the world are either 
offering or exploring the option to offer MOOCs for a certification or credit. 

c. MOOC Taxonomy and Instructional Models 

As the number of students and course offerings including institutions that offer MOOCs continues to 
grow; the MOOC typology has evolved and different variants of MOOC have emerged. To analyze the 
emerging assessment trends in MOOCs, it is important to analyze MOOC taxonomy including the 
different MOOC types that have shaped the evolving landscape in recent years. A study of these 
different MOOC models can also help to address wide ranging issues that have long been plaguing 
MOOCs. These include, but are not limited to engaging and motivating the learners; low completion 
rates; and effectively accommodate the diverse needs of learners around the world. The MOOC 
taxonomies have direct or indirect ramifications in the design of assessment techniques for MOOCs. 
MOOCs today include as many as 10 million students from around the world (Shah, 2013). MOOCs 
have been primarily categorized into two types - xMOOCs and cMOOCs. The xMOOCs follow the 
classroom model with instructor-led video lectures on course topics. In contrast, cMOOCs rely on the 
connectivist pedagogy and learner networks to create new knowledge (Downes, 2012). cMOOCs 
provide more learner autonomy and control through the free use and selection of technology to 
accomplish learning objectives (Downes, 2012). It is important to note however, that most MOOCs are 
variants of xMOOCs and rely on the traditional classroom-based model for delivery of instruction. In 
recent times however, there have been different variants of MOOCs that adopt the hybrid version 
combining components of both xMOOC and cMOOC. Detailed explanation and key differences of these 
MOOC types - xMOOC, cMOOC, and hybrid MOOC and their taxonomy are discussed later in the 
following sections. 

Big Open Online Course (BOOC) 

Funded by a $50,000 Google grant to Indiana University, professor Daniel Hickey launched BOOC in 
September 2013 (Google funds BOOC, 2013). The BOOC follows a hybrid format since it combines 
distributed learning (cMOOC) with personalized feedback (xMOOC). The BOOC is based on the theory 
of situated cognition popularized by Jim Greeno focusing on contextual learning (Hickey, 2013). 

The BOOC, titled "Assessment Practices, Principles and Policies" focuses on classroom assessment 
practices. The BOOC is targeted at educators to foster dialog about assessment practices, techniques 
and administrative accountability. A controversial concept taught in the course by experts was Value- 
added Modeling (VAM) which focused on making teachers more accountable. The concept stressed on 
learning outside the classroom (Hickey, 2013). The BOOC required participants to post their 
reflections on complex concepts on WikiFolio in order to be considered for awarding digital badges. At 
least one student endorsed the Wikifolio for the award of a badge (Hickey, 2013). The endorsed pages 
were visible on the student's homepage for engaging and motivating the learner. 

The BOOC had a unique intuitive interface with clear headings and description of sub-topics each week 
for the discussions to remain anchored around the weekly topics. The students were required to 
discuss complex topics from the textbook and reflect on how they would apply these concepts and 
ideas to their daily future job role. An essential assessment feature of the course was to revisit 
previous week's posting of their classmates for at least one endorsement from their peers. This form 
of participatory assessment paired students in groups to reflect on previous week's assignment 
(Hickey, n.d.). The students were Badges were awarded to successful students who were motivated to 
showcase them on social media platforms highlighting their accomplishment to potential job 
employers. 

The BOOC utilized Google's Coursebuilder platform to encourage interaction by using participatory 
assessment and also to investigate if this technique is scalable for a class of 500 students (Hickey, 
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n.d.). The BOOC also aims to study how students who are awarded digital badges use them by 
displaying and sharing them across social media platforms. 

Distributed Open Collaborative Course (DOCC) 

DOCCs include 17 participating universities and colleges. The class size is limited up to 30 students 
and available for free to students around the world with an option for credit (Jaschik, 2013). DOCC 
follows the hybrid pedagogy which includes videos lectures by experts (xMOOC) based on weekly 
themes by the participating universities. The DOCC recognizes that knowledge is distributed in the 
network (cMOOC). The learning materials, readings and assignments are distributed and based on the 
syllabus, grading rubric and customized for the target student population at each participating 
university (Jaschik, 2013). DOCCs aim to distribute pedagogical expertise with the participating 
universities for accomplishing the learning outcomes. The model emphasizes distributed learning, 
knowledge building, and sharing within a network. The model aims to stress learning beyond the 
classroom and encourages learner participation by collaborative sharing of knowledge, ideas, learning 
materials and assessments across the 17 participating universities. 

The DOCC instructional model involves a constructive dialogue on feminism and technology delivered 
in a video format featuring experts in the field. The 12 videos can be accessed by anyone with internet 
access and discuss female perspectives on the use and application of technology. The participants are 
provided an opportunity to actively engage and discuss the weekly topics on various electronic and 
digital media. The experts join in the online forums in brainstorming and sharing of ideas, and building 
of collective knowledge. The DOCC instructional model aims to foster self-reflection though the 
utilization of a variety of learning resources at the 17 universities. The instructional strategies and 
learning activities utilized by the experts are distributed across the 17 universities and encourage 
active learner engagement and participation through the use of topics that are anchored around 
feminism and allow the learner to post their comments in online forums and engage in active 
participation with the experts in the course. 

Little Open Online Course (LOOC) 

University of Maine in the US is the first university to offer a LOOC. A LOOC is hosted on a college LMS 
and students must login with their university IDs. LOOC is typically open to 5-7 non-registered 
students for free along with 15-20 regular college or university students who pay tuition and receive 
credit. All the students receive direct and personalized feedback from the instructor (Kolowich, 2012). 
Currently, participation by non-registered students is limited to local inhabitants of Maine. 

The LOOC instructional model follows the typical US higher educational model by providing an 
opportunity for the learners for a free-trial to preview the course before confirming their participation. 
Once the course begins, the non-registered participants have 1-2 weeks till the add/drop period to 
decide if they actually want to continue to participate in the course. Upon confirmation, they are 
treated as regular students in the class and must pay tuition to register for the required number of 
LOOC credits. Alternatively, the learners can also continue to participate in the course and submit 
assignments for a personalized feedback from the instructor. The non-registered students will have 
access to the same learning materials and resources. However, the participants must pay to receive a 
credit for successfully completing the course (Kolowich, 2012). 

The LOOC delivery model follows the popular xMOOC format with the use of instructor videos, 
podcasts and participation via the course discussion board. Currently, research is still evolving on the 
learning outcomes and the learner participation levels, and engagement techniques in a LOOC. 

The Community College of Vermont (CCV) is the second institution after University of Maine that has 
offered two LOOCs since Fall 2013. On the Hunt for the Perfect Job, a five-week LOOC was offered in 
October 2013 at CCV and targeted at the small local Vermont population (Larkin, 2013). The second 
LOOC, Money Smarts, was another five-week course that was offered not-for-credit in March 2014 
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(Eddy, 2014). 

Massive Open Online Research (MOOR) 

MOOR is yet another variant of a MOOC. University of California, San Diego will be the first to offer a 
MOOR in Bioinformatics Algorithms - Part 1. The eight week course was offered in October 2013 and 
includes research projects (Bolkan, 2013). The course is open to participants with a basic knowledge 
of programming. Those participants who do not have proficiency in a programming language are 
required to complete a basic foundation course. MOOR is targeted at learners who want to transition 
into research. The participants in the course work on research projects under the guidance and 
mentorship of top bioinformatics scientists from around the world (Bolkan, 2013) 

The MOOR instructional model follows the popular xMOOC model that includes lecture videos by the 
instructors. The course materials include an e-textbook written by the course instructors. The text 
includes exercise and problem solving which can also be accessed on the course website. Each chapter 
in the text builds on basic concepts and ideas to introduce complex topics and assesses mastery of 
concepts learning through coding challenges and homework assignments. The course included 24x7 
support by teaching assistant for grading assignments. 

MOOR relied on innovative technologies for learner participation and engagement by integrating 
theoretical concepts with interactive problem solving and quizzes. MOOR also takes exiting skills into 
consideration since a basic knowledge of programming basics is necessary for a successful 
accomplishment of course learning objectives. 

The MOOR model incorporates Vygotsky's Zone of Proximal Development (ZPD) by scaffolding the 
learner into learning advance programming skills and mastering of concepts. The use of innovative 
technology helps to foster learner engagement and participation. 

Small Private Online Course (SPOC) 

The term "SPOC" was coined in 2013 by Professor Armando Fox at UC-Berkeley (Fox, 2013). SPOCs 
debuted at Flarvard Law School in early 2013. Approximately 500 students were selected from a total 
of 4,100 applicants (Conway, 2013; Patel, 2013). The selected applicants had to satisfy minimum 
requirements for English to be eligible for enrollment in the SPOC. The SPOC enrollees included 
diverse set of learners that includes high school students to practicing lawyers with or without any 
background knowledge of law. The enrollees are either law professionals or have interest in related 
fields that touch upon or have legal implications (Patel, 2013). 

A SPOC is modeled on xMOOC and includes instructor videos, interactive assignments and discussion 
groups. SPOCs follow the "flipped classroom" model of instruction (Coughlan, 2013). The 500 online 
students take the course along with students enrolled at Flarvard. The online students are mentored 
by enrolled campus students and must complete all the requirements of the course expected of a 
regular campus students. The enrollees discuss learning materials in a video chat moderated by 
former students or on-campus students who are currently enrolled in the course. The SPOC also hosts 
webcast with guest speakers during the 12-week course (Patel, 2013). Students who do not perform 
at an optimal level defined by a rubric are dropped from the course (Shimabukuro, 2013). SPOCs 
make it convenient to admit students in a cohort for promoting learner-learner interaction and 
improving learning outcomes. 

The SPOC instructional model aims to better the abysmal attrition rates of below 10% for MOOCs 
(Flasmi, 2013). Flarvard has launched two more SPOCs in Fall 2013 since the first SPOC in 2013. The 
SPOCs will have limited enrollment like the first SPOC and include discussion boards for fostering 
learner interaction. 

Synchronous Massive Open Online Courses (SMOC) 

Limited to a total of 10,000 students, the University of Texas at Austin launched the first SMOC in fall 
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2013. The Introduction to Psychology at the university includes live lectures by the instructors to not 
only the students who are enrolled in the course at the university but the lectures are also available in 
real time to all the enrollees around the world (Straumsheim, 2013). The SMOC topic includes how 
students learn and offers three transferable credits. The SMOC is available to all the participants at a 
registration fee of $550. The participants will need to be available for the live lectures (Straumsheim, 
2013). 

The instructors experimented with several techniques such as adaptive learning, group chats and 
classroom discussions before coming up with the SMOC instructional model. The SMOC does not have 
any final or mid-term examination. The students are assessed on the basis of group participation and 
learning activities (UEX, 2014). The data from the course shows gains in academic achievement and 
improved learning outcomes compared to the face-to-face class (Straumsheim, 2013). The online 
students in the course do not have to pay out-of-state tuition. The students earn a course credit upon 
a successful completion of the course. 

SMOC follows the popular xMOOC model. The learning materials include free web resources and online 
readings. Live lectures are used to engage the learners and learner participation is encouraged via 
participation in discussion forums. The classes are divided into small groups and monitored by TAs 
who are former students of the course. Having the same students in the same group throughout the 
semester help create a sense of community and promotes sustained interaction between the students 
(Straumsheim, 2013). To improve learning outcomes, the instructors collect data from the course on a 
regular basis (Kerr, 2013). 

III. MOOC Assessment Models and Emerging Technologies 

Huge enrollments in MOOCs make it impossible to manually grade and provide feedback to learners. 
There are several emerging tools and technologies that are being leveraged to assess learning 
outcomes in a MOOC. These technologies can also be utilized to design and develop a MOOC with 
built-in features to measure learning outcomes. As with any learning environment, the design of 
MOOC must foster online student engagement and learning through the use of interactive pedagogy 
that provides greater learner autonomy and control. The pedagogy must support assessment and 
measuring of learning outcomes. The following section discusses emerging technologies for 
assessment with case study examples of their application and use in a MOOC. 

a. A Learning Analytics xMOOC Case Study: Circuits and Electronics (6.002x) 

MIT's first for-certificate MOOC, "Circuits and Electronics (6.002x)" is an exemplary case study of 
applying learning analytics for assessment and accreditation in MOOCs. The first phase of the study 
examined how the students utilized the various course components in the course and how their 
behavior differed when completing the course requirements. The detailed data includes learner 
utilization of course components including their demographics. The "Circuits and Electronics (6.002x)" 
course components include homework, lab, mid-term and final exam; lecture video; lecture question; 
wiki; tutorials; and discussion forums. The second phase of the study profiled student demographics 
and learner characteristics including the factors that contributed to their persistence with the course. 
Finally, the third part of this study analyzes the student interactions on the discussion forum (Breslow, 
Pritchard, DeBoer, Stump, Ho, & Seaton, 2013). The data from the course also analyzed student 
persistence in terms of what contributed or hindered their achievement. Students IP addresses were 
recorded to identify their geography and location. The course data was collected using clickstream and 
is based on approximately 250 million interactions of all the enrolled students in various course 
components (Breslow et. el., 2013). 

"Circuits and Electronics (6.002x)" utilized the popular xMOOC model, also known as an extension of 
classroom, for the MOOC design and delivery. The data from the MOOC shows that nearly 5% 
participants in the course watched video lectures by the instructor but nearly 80% completed the 
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required lab and homework for a certificate (Seaton, 2014). The participants actively engaged and 
participated in course discussions on the discussion forum. The participants used the forum to ask 
queries and get help for completing the lab and homework assignments. The forums supported learner 
interaction and the social component of the course to get assistance for achieving their learning goals. 
The data from the course shows heightened student activity and interaction as the deadline for the 
submission of the homework assignments approached. Among the course components, maximum 
time-on-task was recorded for completing homework and lab assignments. Maximum time was spent 
on viewing lecture videos. Students who viewed existing discussion threads without any new postings 
accounted for 90% of the total time spent on the discussion forum (Breslow et. el., 2013). 

The data from the course provides valuable insight in designing assessments for students spread 
across the globe. The "Circuits and Electronics" (6.002x) student population included students from US 
and 193 other countries around the world. Of the 7,161 students who completed the end of the course 
survey, 6,381 earned a course certificate (Breslow et. el., 2013). 

b. A Personal Learning Network (PLN) cMOOC Case Study: PLENK2010 

MOOCs provide a great learning platform to experiment with emerging technologies for course design, 
delivery and assessment. One such MOOC which utilized emerging technologies is Personal Learning 
Environments Networks and Knowledge (PLENK2010). 

PLENK2010 utilized the cMOOC or Connectivist model for course design and delivery. The course 
encouraged learner participation by creation and sharing knowledge though the use of social media 
tools. The participants in PLENK2010 created their own Personal Learning Networks (PLN). Second Life 
and Facebook groups were created to participate in course discussions (Kop, et. el., 2011). The course 
combined formal and informal learning to promote learner engagement and interaction to measure 
learning outcomes. 

PLENK2010 included live sessions and provided an opportunity to the learners to interact via chat 
messenger. The live sessions provide a great opportunity for the learners to express their thoughts 
and ideas while also listening and actively engaging in a dialogue with the expert and other 
participants. The tools used by participants in PLENK2010 were specific to their needs, personal 
learning preferences and learning goals. Their choice was also determined by language preferences 
and technology competencies (Rodriguez, 2012). 

The Connectivist pedagogy empowered the learners by co-creation of knowledge by allowing them to 
interact, communicate, and collaborate with their peers to share knowledge and learning resources 
through the use of social media. The use of social media for learning gives learners a robust 
"backchannel" to connect and interact with others. SO, while live sessions remain the primary source 
of learning form an expert, the backchannels allow the learners to engage in dialogue and select their 
medium of preference. The learners can be selective in their choice and use of these backchannels to 
connect with others. They can control and mange their subscriptions, and carefully select who they 
follow for learning and achievement of their learning goals. A cMOOC thus puts the learner in control 
of a dynamic learning environment where there is a continuous inflow of information and creation of 
new knowledge. 

An analysis of the data from this MOOC shows active participation by learners in the course 
components. The course components include Twitter handle #PLENK2010, live Elluminate sessions, 
blogs, and discussion posts. PLENK2010 course surveys show an active participation by the learners to 
produce digital artifacts for completing assessments (Kop, Fournier, & Mak, 2011). 

c. A Mobile Learning cMOOC Case Study: MobiMOOC 

Data collected from another cMOOC titled MobiMOOC shows high levels of learner interaction in the six 
week course. MobiMOOC used connectivist pedagogy for course design and delivery. The learners in 
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MobiMOOC utilized mobile technologies for accessing course content, knowledge creation and sharing 
within the network. Convenience of time and space was the predominant factor for the use of mobile 
technology and devices (Rodriguez, 2012). The course participants created a Google group to 
participate in discussions. The data from the course shows that 1,827 discussion threads were created 
for course. There were total 1,123 tweets for the #mobimooc hashtag. The data also recorded text 
and voice messages for assessment purposes. Highest participation levels were reported for the live 
MobiMOOC lectures on Monday. Around 335 mobile learning links were shared on social bookmarking 
sites (de Waard, Abajian, Gallagher, Hogue, Keskin, Koutropoulos, & Rodriguez, 2011). The learners in 
the course took control of their learning and building their own mobile learning projects through 
knowledge creation and personalized learning. The course attracted over 300 unique visitors from 
nearly 30 countries. The course participants used mobile devices to interact with course materials 
citing independence of location (de Waard, et. al., 2011). MOOCs provide tremendous opportunity to 
explore emerging technologies for research and practice of assessment to achieve learning outcomes. 

d. Digital Badges 

"Open Badges" ( http://openbadqes.org/ ) is a digital badges project launched by Mozilla Corporation in 
September 2011. The project aims to reward accomplishments and online learning achievement by 
fostering life-long learning. Digital Badges extend recognition of student learning and achievement 
beyond the classroom. The learning outcomes can include learner engagement with the content and 
course components to complete learning activities, assignments, and accretion of specific knowledge 
and skills. The "Open Badges" architecture uses three interfaces - Badge Issuer; Badge Earner; Badge 
Disptayer ( http://openbadqes.org/ . Digital badges can be integrated with MOOC platforms to award 
certification for successful completion of the course and reward the learners for their accomplishment 
by issuing a badge that can be displayed across multiple platforms on the web. The Badge Issuer or 
the MOOC hosting platform can assess learners for their discrete skills and knowledge. The Badge 
Earner or the learner can display these badges on social media or Badge Disptayer platforms such as 
Linkedln to showcase their talent and skills. 

University and institutions have begun experimenting with similar platforms for issuing badges to 
recognize learning and achievement. Linkedln, the professional networking site users can now add 
their completed MOOC courses on Coursera to their Linkedln profile (Baird, 2013). Indiana University's 
BOOC - "Assessment Practices, Principles and Policies" uses Google's Course Builder platform to 
provide four different types of badges for the BOOC participants. These include Assessment Expertise 
badge for peer-review of wikifolios; Assessment Expert badge for earning the three expertise badges 
including success in the final exam; Leader badge for maximum peer-promotions; and Customized 
Assessment Expert badge for successfully completing a term paper and demonstrating local expertise 
about assessment (Hickey & Kelley, 2013). Another institution, Purdue University, uses its own badges 
system "Passport" to issue badges based on the successful accomplishment of course objectives 
(Tally, 2012). 

Recently CourseSites, the Blackboard MOOC hosting platform launched "Badges as New Currency for 
Credentials". This 6-week MOOC launched in September 2013 aims to investigate the use and 
application of digital badges for credentialing skills and knowledge of learners. The findings from the 
study aim to benefit institutions especially MOOC providers as they begin to integrate badges for 
assessment and accreditation (WCET launch MOOC, n.d.). 

e. Adaptive Assessment 

MOOCs with huge enrollments make it impossible to afford the luxury of learner-instructor interaction. 
MOOCs therefore, depend on automated assessments to not only grade but also provide automated 
feedback to the learners. In that sense, automated assessments therefore provide the learner a 
unique advantage of anywhere, anytime grading and feedback (Reeves, 2000). Research has shown 
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that assessments based on Item Response Theory (IRT) can be used effectively in online assessments 
(Chen, Lee, & Chen, 2005; Baylari & Montazer, 2009). These tests are designed to automatically 
adapt to student learning and ability to measure learner performance and learning outcomes. The 
tests include different difficulty levels and based on the response of the learner to each test item, the 
difficulty level decreases or increases to match learner ability and potential Meyer, & Zhu, 2013). The 
final score is based on the difficulty levels of the correct answers (Challis, 2005). The advantage of 
using adaptive assessments is that based on the final score, course content can be tailored to meet 
individual learner needs (Baylari & Montazer, 2009). Since MOOC participants are not limited to a 
geographical location and include different background knowledge and skills, designing learning 
modules that use adaptive assessment can go a long way in making learning relevant and meaningful 
to the participants who enroll in a MOOC. 

f. Automated Assessments 

Two popular forms of automated assessments that have been tested and are currently being used for 
MOOC assessments are the Automated Essay Scoring (AES) and UCLA's Calibrated Peer Review™ 
(CPR) (Shermis, Burstein, Higgins, & Zechner, 2010). AES can check student writing assignments for 
grammar; style; complexity; vocabulary usage and report on the content alignment with the overall 
theme of the essay. This helps to provide prompt feedback to students regarding their writing. Early 
research on automated MOOC assessments or peer-grading has shown positive results (Kolowich, 
2013). In a recent survey, MOOC faculty expressed satisfaction in using automated assessment 
techniques and found it to be reliable (Kolowich, 2013). There is no unanimity when it comes to 
evaluating student writing with CPR since some studies have not shown any improvement in student 
achievement by improving their essay scores. Other studies on the other hand, have reported gains in 
student attitudes and student belief that CPR helps them to better evaluate and improve their writing 
skills. 

While there is no conclusive evidence to show that either AES or CPR can accurately assess various 
other forms of writing such as creative writing, the adoption of machine grading by the two popular 
MOOC platforms EdX and Coursera for written essays makes it possible to provide categorical 
feedback to students and improve their writing skills (Balfour, 2013). When compared to AES, CPR is 
suitable for large sections and can give personalized feedback to students. The preference of one over 
the other would largely depend on the course structure, pedagogy, student expectations, and learning 
outcomes. MOOCs might use AES initially for formative assessment and CPR for summative evaluation 
(Balfour, 2013). This model also makes it convenient to correct any mechanical problems encountered 
in the writing process before using CPR for the final evaluation. Research on students using CPR has 
shown that students' benefit more by exposure to quality writing and creative thinking. The use of 
both AES and CPR for grading student essays could produce prolific writers than if they were grade 
manually (Balfour, 2013). 

g. Recognition of Prior Learning (RPL) 

Recognition of Prior Learning (RPL) has been one of the popular assessment techniques in Open 
Education Resources (OER). Recognition of Prior Learning (RPL) requires students to submit portfolios 
followed by a summative assessment to demonstrate their learning for the purposes of a certification 
or credit (Conrad, Mackintosh, McGreal, Murphy, & Witthaus, 2013). Institutions that support RPL 
have the necessary resources to support certification and credit through RPL. Since MOOCs include 
course offerings from reputed universities around the world, integrating RPL into MOOC assessments 
looks very much feasible with support from the course offering universities and institutions. Since 
MOOCs include tens and thousands of participants from around the world, those who enroll in a MOOC 
for a credit or certification will find this form of assessment especially useful in pursuing their learning 
goals. In addition to providing participants with an opportunity to assess their prior learning and skills, 
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institutions can also leverage RPL for formative and summative assessments. 

IV. Conclusion 

Assessment in MOOCs is about supporting student learning and achievement. MOOCs are ultimately a 
learning environment that have found mass appeal and favor with the hundreds and thousands of 
learners. MOOCs bring a new dimension to assessing such large number of learners. The automated 
assessments that have evolved in recent years are specifically targeted to assess and evaluate the 
large enrollments since it is not possible to manually grade and provide feedback to all the learners. 
Assessment techniques that certify both formal and informal forms of learning in a MOOC can also 
help to tackle the most nagging problem - low completion rates. Assessment in a MOOC does not 
necessarily have to be about course completion. Learners can be assessed on time-on-task; learner- 
course component interaction; and a certification of the specific skills and knowledge gained from a 
MOOC. While not the primary aim, these assessment techniques will provide an added incentive for 
the learners to persist and complete the MOOC. Ultimately, the satisfaction gained from completing 
the course can be potential indicator of good learning experiences. Conversely, enhancing the learning 
experience can contribute to improving the MOOC completion rates. As a result, assessment 
techniques that permit customization of content catered to the individual learner can track learner 
behavior and predict learning outcomes. Such a technique will further assist in developing and refining 
assessment procedures for improving learning outcomes. 
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